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(57) Abstract: A system and method for manipulating a set of images of a static scene captured at different exposures (i.e., "brack- 
eted" images) to yield a composite image with improved uniformity in exposure and tone. This is generally achieved by analyzing 
a set of bracketed images using a multi-dimensional histogram and merging the images via an approach that projects pixels onto a 
curve that fits the data (400-406). However, it has been found that the desired composite image can also be produced by slimming 
the pixel brightness levels across the multiple images (706), followed by an equalization process. One possible equalization process 
involves simply averaging the summed pixel brightness values by dividing the summed value of each pixel set (i.e., groups of cor- 
responding pixels from the bracketed images) by the number of bracketed images. An even better result can be achieved using a 
histogram equalization process (710-720). 



WO 00/78038 PCT/US00/15902 

A SYSTEM AND PROCESS FOR IMPROVING THE UNIFORMITY OF THE 
EXPOSURE AND TONE OF A DIGITAL IMAGE 

BACKGROUND OF THE INVENTION 

5 

Technical Field: 

The invention is related to image processing in general, and more 
particularly, to a system and process for generating a composite image from a set 
10 of digital images of a static scene captured at different exposure levels. The 
composite image exhibits an improved uniformity in both exposure and tone in 
comparison to the set of input images. 

Background Art: 

15 

Digital cameras suffer from several deficiencies when compared with 
traditional photographic film. Among these are a lower dynamic range and a 
resulting inconsistency in the uniformity of both exposure and tone across an 
image captured by current digital cameras. 

20 

Figs. 1A through 1C present digital images which will be used to explain 
the dynamic range problem. Figs. 1 A through 1 C show three images of an office 
desk and window, taken at different exposures. Specifically, these images were 
captured with a Kodak DCS-40 camera, by adjusting the exposure up and down 

25 by two "stops". Fig. 1 B represents an image taken at what was considered the 
"proper" exposure, whereas, the image of Fig. 1 A is considered to be 
"underexposed" and the image of Fig. 1 C is considered "overexposed". Notice 
how the "underexposed" image of Fig 1 A shows better detail for the outside 
elements (e.g., trees, sky), while the "overexposed" image of Fig. 1c shows the 

30 interior element (e.g., portraits) better. Also note that while the image of Fig. 1 B 
was taken at what would normally be considered the proper exposure, it still 
contains areas that appear to be underexposed or overexposed. 
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DISCLOSURE OF THE INVENTION 



The present invention relates to a system and process for manipulating a 
5 set of images of a static scene captured at different exposures (i.e., "bracketed" 
images) to yield a composite image with improved uniformity in exposure and 
tone. 



Bracketing is term used in photography meaning to take photographs of 
l o the same scene at multiple exposure setting in hopes of obtaining one with the 
optimum exposure level. This term is borrowed in the context of the present 
invention to describe a process for overcoming the aforementioned shortcomings 
of cameras in respect to dynamic range. In general, the invention involves 
combining multiple images having different exposure levels in such a way as to 
15 create a "better looking" image. It is noted that for the purposes of this 
description, the term "bracketed" images will mean a set of images having 
different exposure levels. 



A better looking image can be obtained because it has been found that 
20 objects depicted in an image of a scene tend to vary in detail depending on the 
exposure setting. For example, in the previously described experiment it was 
found that objects in the background of an image appeared in greater detail in an 
underexposed image, while objects in the foreground showed more detail in an 
overexposed image. It is believed these inconsistencies resulted in a non- 
25 uniformity in the exposure and tone associated with the various pixels making up 
different parts of the image. The goal of the subject invention is to produce a 
single image from images of a static scene taken at different exposures, which 
captures to some extent, the portions of each image providing the best exposure 
and tone characteristics. 

30 

In general, the aforementioned goal can be achieved by analyzing a set of 

2 



WO 00/78038 PCT/US00/1 5902 

bracketed images using a multi-dimensional histogram and merging the imag s 
via an approach that projects pixels onto a curv that fits the data. However, it 
has been found that the desired composite image having improved exposure and 
tone can also be produced in a simpler manner by summing characteristic pixel 
5 values across the multiple images, followed by an equalization process. The 
characteristic pixel values are preferably any appropriate measure of the pixel 
brightness level (also known as the luminous intensity value) exhibited by a pixel. 
For example, if the images are black and white, the pixel gray level could be 
used. If color images are involved, the Y-luminance channel could be used. 

10 

One possible equalization process would be to simply average the chosen 
characteristic pixel values. This is accomplished by dividing the summed value of 
each pixel set by the number of images in the set of images. It is noted that a 
pixel set is made up of all the corresponding pixels from the bracketed images, 
15 where corresponding pixels are defined as the pixels that represent the same 
portion of the depicted scene. While this simple averaging method does produce 
a better looking picture, an even better apparent dynamic range can be achieved 
using a histogram equalization process. 

20 In essence, this histogram equalization involves creating a count of the 

number of pixels sets having the same summed brightness level. From this 
count, a cumulative distribution function is computed and normalized to a 
maximum value corresponding to the maximum summed brightness level. The 
cumulative distribution function is then used to determine new pixel brightness 

25 levels to use in generating the composite image. Specifically, the normalized 
cumulative count value that corresponds to the summed brightness value 
associated with each pixel set is identified and divided by the number of images 
in the set of bracketed images to produce a new pixel brightness value 
associated with each pixel set. The composite image is generated by using the 

30 new pixel brightness value associated with a pixel set as the pixel value for a 
pixel in the composite image corresponding to the location of the pixels of the 
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associated pixel set in the scene depicted by the images. 



A further refinement can be implemented by employing a partial 
equalization approach. Partial equalization can be described as the blending of 
5 the aforementioned normalized cumulative distribution function with a straight line 
function. The degree of equalization of the image is controlled by the percentage 
of blending of the straight line function with the cumulative distribution function 
(i.e. any blend between 0% and 100% is possible). 

10 In addition to the just described benefits, other advantages of the present 

invention will become apparent from the detailed description which follows 
hereinafter when taken in conjunction with the drawing figures which accompany 
it. 



15 



BRIEF DESCRIPTION OF THE DRAWINGS 



The specific features, aspects, and advantages of the present invention 
20 will become better understood with regard to the following description, appended 
claims, and accompanying drawings where: 

FIGS. 1 A through 1C are images depicting an office scene captured at 
different exposure settings. 

25 

FIG. 2 is a diagram depicting a general purpose computing device 
constituting an exemplary system for implementing the present invention. 

FIG. 3 is a block diagram of an overall process for generating a composite 
30 image having a more uniform exposure and tone in comparison to any image in a 
set of bracketed images used as a basis for the composite image. 
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FIG. 4 is a block diagram of a process for accomplishing the composite 
image generation module of the overall process of Fig. 3. 

5 FIGS. 5A through 5C are graphs plotting 2-D histograms based on the 

brightness values of pixels in pairs of the images of Figs. 1 A through 1 C. 

FIG. 6 is a composite image produced from the bracketed images of the 
office scene of Figs. 1 A through 1C that exhibits a more uniform exposure and 
10 tone than any of the bracket images, wherein an averaged summed brightness 
equalization process was employed to generate the composite image. 

FIGS. 7A and 7B are block diagrams of an alternate process for 
accomplishing the composite image generation module of the overall process of 
15 Fig. 3. 

FIG. 8A is a graph plotting the histogram for the summed brightness levels 
computed for the images of Figs. 1A through 1C. 

20 FIG. 8B is a graph of a cumulative distribution function based on the 

histogram of Fig. 8A. 

FIG. 9A is a graph of a cumulative distribution function similar to Fig. 8B. 

25 FIG. 9B is a graph of a straight line function. 

FIG. 9C is a graph of a blended distribution function produced by blending 
of the cumulative distribution function of Fig. 9A with the straight line function of 
Fig. 9B. 

30 

FIG. 1 0A is a composite image produced from th bracketed images of the 

5 
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office scene of Figs. 1A through 1C that exhibits a more uniform exposure and 
tone than any of the bracket images, wherein a full histogram equalization 
process was employ d to generate the composite image. 

5 FIG. 1 0B is a composite image produced from the bracketed images of the 

office scene of Figs. 1 A through 1C that exhibits a more uniform exposure and 
tone than any of the bracket images, wherein a partial (i.e., 50 %) histogram 
equalization process was employed to generate the composite image. 

10 

BEST MODES FOR CARRYING OUT THE INVENTION 

In the following description of the preferred embodiments of the present 
invention, reference is made to the accompanying drawings which form a part 
15 hereof, and in which is shown by way of illustration specific embodiments in which 
the invention may be practiced. It is understood that other embodiments may be 
utilized and structural changes may be made without departing from the scope of 
the present invention. 

20 Fig. 2 and the following discussion are intended to provide a brief, general 

description of a suitable computing environment in which the invention may be 
implemented. Although not required, the invention will be described in the 
general context of computer-executable instructions, such as program modules, 
being executed by a personal computer. Generally, program modules include 

25 routines, programs, objects, components, data structures, etc. that perform 
particular tasks or implement particular abstract data types. Moreover, those 
skilled in the art will appreciate that the invention may be practiced with other 
computer system configurations, including hand-held devices, multiprocessor 
systems, microprocessor-based or programmable consumer electronics, network 

30 PCs, minicomputers, mainframe computers, and the like. The invention may also 
be practiced in distributed computing environments where tasks are performed by 
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remote processing devices that are linked through a communications network. In 
a distributed computing environment, program modules may be located in both 
local and remote memory storage devices. 

5 With reference to Fig. 2, an exemplary system for implementing the 

invention includes a general purpose computing device in the form of a 
conventional personal computer 20, including a processing unit 21 , a system 
memory 22, and a system bus 23 that couples various system components 
including the system memory to the processing unit 21 . The system bus 23 may 

10 be any of several types of bus structures including a memory bus or 

memory controller, a peripheral bus, and a local bus using any of a variety of bus 
architectures. The system memory includes read only memory (ROM) 24 and 
random access memory (RAM) 25. A basic input/output system 26 (BIOS), 
containing the basic routine that helps to transfer information between elements 

15 within the personal computer 20, such as during start-up, is stored in ROM 24. 
The personal computer 20 further includes a hard disk drive 27 for reading from 
and writing to a hard disk, not shown, a magnetic disk drive 28 for reading from or 
writing to a removable magnetic disk 29, and an optical disk drive 30 for reading 
from or writing to a removable optical disk 31 such as a CD ROM or other optical 

2 0 media. The hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 
are connected to the system bus 23 by a hard disk drive interface 32, a magnetic 
disk drive interface 33, and an optical drive interface 34, respectively. The 
drives and their associated computer-readable media provide nonvolatile storage 
of computer readable instructions, data structures, program modules and other 

2 5 data for the personal computer 20. Although the exemplary environment 
described herein employs a hard disk, a removable magnetic disk 29 and a 
removable optical disk 31 , it should be appreciated by those skilled in the art that 
other types of computer readable media which can store data that is accessible 
by a computer, such as magnetic cassettes, flash memory cards, digital video 

30 disks, Bernoulli cartridges, random access memories (RAMs), read only 
memories (ROMs), and the like, may also be used in the exemplary operating 



7 



WO 00/78038 PCT/US00/15902 

environment. 

A number of program modules may be stored on the hard disk, magnetic 
disk 29, optical disk 31 , ROM 24 or RAM 25, including an operating system 35, 
5 one or more application programs 36, other program modules 37, and program 
data 38. A user may enter commands and information into the personal 
computer 20 through input devices such as a keyboard 40 and pointing device 
42. Of particular significance to the present invention, a camera 55 (such as a 
digital/electronic still or video camera, or film/photographic scanner) capable of 

1 o capturing a sequence of images 56 can also be included as an input device to the 

personal computer 20. The images 56 are input into the computer 20 via an 
appropriate camera interface 57. This interface 57 is connected to the system 
bus 23, thereby allowing the images to be routed to and stored in the RAM 25, or 
one of the other data storage devices associated with the computer 20. 

15 However, it is noted that image data can be input into the computer 20 from any 
of the aforementioned computer-readable media as well, without requiring the use 
of the camera 55. Other input devices (not shown) may include a microphone, 
joystick, game pad, satellite dish, scanner, or the like. These and other input 
devices are often connected to the processing unit 21 through a serial port 

20 interface 46 that is coupled to the system bus, but may be connected by other 
interfaces, such as a parallel port, game port or a universal serial bus (USB). A 
monitor 47 or other type of display device is also connected to the system bus 23 
via an interface, such as a video adapter 48. In addition to the monitor, personal 
computers typically include other peripheral output devices (not shown), such as 

2 5 speakers and printers. 

The personal computer 20 may operate in a networked environment using 
logical connections to one or more remote computers, such as a remote computer 
49. The remote computer 49 may be another personal computer, a server, a 
30 router, a network PC, a peer device or other common network node, and typically 
includes many or all of the elements described above relative to the personal 

8 
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computer 20, although only a memory storage device 50 has been illustrated in 
Fig. 2. The logical conn ctions depicted in Fig. 2 include a local area network 
(LAN) 51 and a wide area network (WAN) 52. Such networking environments 
are commonplace in offices, enterprise-wide computer networks, intranets and 
5 the Internet. 

When used in a LAN networking environment, the personal computer 20 is 
connected to the local network 51 through a network interface or adapter 53. 
When used in a WAN networking environment, the personal computer 20 

1 o typically includes a modem 54 or other means for establishing communications 

over the wide area network 52, such as the Internet. The modem 54, which may 
be internal or external, is connected to the system bus 23 via the serial port 
interface 46. In a networked environment, program modules depicted relative to 
the personal computer 20, or portions thereof, may be stored in the remote 
1 5 memory storage device. It will be appreciated that the network connections 
shown are exemplary and other means of establishing a communications link 
between the computers may be used. 

The exemplary operating environment having now been discussed, the 

2 0 remaining part of this description section will be devoted to a description of the 

program modules embodying the invention. 

In general the present invention relates to a system and process for 
creating a composite image from a set of digital images of a static scene. This 

25 composite image exhibits an improved uniformity in both exposure and tone 
throughout the composite image in comparison to the original set of images. Fig. 
3 presents a high-level outline of the steps required to produce the 
aforementioned composite image. As can be seen there are two basic steps in 
the task. In the first step 300, a set of digital images of a static scene is created. 

30 Each of these images is captured at a different exposure level. As mentioned 
previously the images are collectively referred to as "bracketed" images. The 
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second step 302 is then to generate the desired composite image from the set of 
bracketed images by setting each pixel in the composite image to a characteristic 
pixel value based on the distribution of this characteristic pixel value in the pixels 
of each image in the set of bracketed images. As indicated previously, the 
5 characteristic pixel values are preferably the pixel brightness levels (also known 
as the luminous intensity values) exhibited by a pixel as the brightness directly 
influences the exposure and tone of the pixel. 

1 .0. Fitting A Multi-Brightness Curve 

10 

The first approach according to the present invention for creating the 
aforementioned composite image involves analyzing the set of "bracketed" 
images using a multi-dimensional histogram and projecting pixels onto a curve 
that fits the data. The flow chart presented in Fig. 4 outlines this approach. In 
1 5 the first step 400 of the procedure, the brightness values associated with each 
pixel in each of the bracketed images are used to generate a n-D histogram, 
where n corresponds to the number of images in the set of bracketed images. 

The distribution of pixel values across the set of bracketed digital images 
20 could be visualized by examining a plot of the just described histogram. For 
example, referring once again to the three images shown in Figs. 1A through 1C, 
a 3-D histogram could be plotted for examination. However, it is believed an 
even simpler way of presenting the distribution data would be to plot a series of 
2-D histograms of pairs of images in the set. Three such 2-D histograms are 
25 presented in Figs. 5A through 5C, respectively. The histogram shown in Fig. 5A 
plots pixels values associated with the image of Fig. 1 A (horizontal axis) against 
the values exhibited by correspondingly located pixels in the image of Fig. 1 B 
(vertical axis). The histogram shown in Fig. 5B plots pixels values associated with 
the image of Fig. 1 B (horizontal axis) against the values exhibited by 
30 correspondingly located pixels in the image of Fig. 1C (vertical axis). And finally, 
the histogram shown in Fig. 5C plots pixels values associated with the image of 

10 
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Fig. 1 A (horizontal axis) against the values exhibited by correspondingly located 
pixels in the image of Fig. 1C (vertical axis). Each of these histograms was 
generated by taking pairs of pixels from the two images being consideration, and 
plotting their respective red, green and blue pixel levels (i.e., level 0 through level 
5 255) on the graph. 

The same trend can be seen in the histograms of all three pairs of images. 
The values in the two images are related by a linear slope for underexposed and 
properly exposed pixels, and saturate earlier in the more exposed image. 

1 o Surprisingly, the is not a strong tail (saturation) effect in the dark regions. It is 

believed this occurs because, for these images, there were no parts of the scene 
that were below the sensitivity range of the camera. 

From the three 2D histograms in Figs. 5A through 5C, it is possible to get a 
is clear idea of what the full 3D histogram would look like. Specifically, the data 
points would cluster around a space curve that has a roughly linear behavior in 
one region, and clings to hypercube faces or edges in other places. Identifying 
this curve is the next step in the process of creating a composite image from the 
bracketed images. Thus, referring once again to Fig. 4, the next step 402 in the 

2 o process is to fit a curve to the data points of the previously constructed n-D 

histogram. This step of the process can be achieved in a number of ways. For 
example, it is possible to construct a spline curve to model the ideal curve 
associated with the data. The spline itself could be computed using any 
appropriate conventional dynamic curve-fitting algorithm, or perhaps modeled as 

25 a combination of elongated ellipsoids. However, to simplify the curve fitting 
process, it would also be possible to project all data points onto a series of 
parallel planes, and compute the centroid (and distribution, if desired) of each 
plane. The centroids of the planes would then define the curve fitting the data. 
One potential set of planes that could be used for this purpose are the so-called 

30 summed brightness planes. These summed brightness planes represent the sum 
of all pixel lev Is across the n images. A nice property of summed brightness is 
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that it is a monotonic function of the true luminance, as long as each of the 
sensor responses is itself monotonic. 

In general, it is possible to produce the composite image from any n- 
5 bracketed images of a scene (i.e., three in the example case) by projecting n-D 
pixel values (taken from the n input images) onto the aforementioned curve, and 
then mapping points along the curve into the final gamut. Thus, the next step 404 
in the process is to project each of the histogram data points onto the nearest 
point of the curve to determine a new brightness value for the pixel associated 
10 with a projected data point. Then, in step 406 a composite image is generated by 
using the new brightness values determined in the projecting step for the pixels of 
any of the bracketed images as a substitute for the original value of a pixel in that 
image. 

15 2.0. Using The Summed Brightness Planes Directly 

Rather than fitting a curve and projecting the pixels onto it, an alternate 
approach would be to use the summed brightness values directly to produce a 
composite image with improved exposure and tone. One simple example of such 

20 an approach would be to sum the pixel values associated with all the 

correspondingly located pixels in n-bracketed images, and then divide by the 
number of images to restore the original gamut and create the desired composite 
image. Figure 6 shows the results of this procedure using the three images of 
Figs. 1A through 1C. Notice how a better apparent dynamic range is obtained 

2 5 with just this simple trick. 

Another approach that employs the summed brightness planes directly 
involves using the summed brightness values as input to a histogram equalization 
process. While the summed brightness approach throws away information that 
30 might be needed to recover true luminance, it is a perfectly acceptable input to a 
histogram equalization process (since it would be expected that pixel values 
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projected onto the space curve and the corresponding summed brightnesses are 
similar). Furthermore, summed brightness is advantageous in that it has the 
effect of weighting "non-clamped 0 pixel values more than the clamped values 
(since those values contribute little to the variation of summed brightness). 

5 

3.0. Histogram Equalization 



Histogram equalization is a very popular technique in image processing. 
Essentially, histogram equalization is a process for mapping pixel brightness 

1 0 values of an image to approximate a uniform distribution. In the context of the 
present invention, the histogram equalization process is preferably performed as 
follows. Referring to Fig. 7 A and 7B, the first step 700 of the process is to select 
one of the images of the previously created set of digital images (i.e., the 
aforementioned bracketed images). One of the pixels of the selected image is 

15 then chosen in step 702. The next step 704 in the process is to identify the pixels 
in the other, non-selected bracketed images that correspond to the chosen pixel 
of the selected image. As indicated previously, these corresponding pixels are 
defined as the pixels in the other images that represent the same portion of the 
depicted scene. The corresponding pixels may simply occupy the same location 

20 in each of the images if the images are in registration. However, even if this is not 
the case, the corresponding pixels can be found using any appropriate 
conventional method typically used for such a task. The chosen pixel and its 
corresponding pixels in the non-selected images will be referred to hereinafter as 
a pixel set. 

25 

Once all the pixels in the pixel set associated with the chosen pixel have 
been identified, their brightness values are summed together to produce a 
summed brightness value or level (step 706). Steps 702 through 706 are then 
repeated for each pixel of the selected image, as indicated in step 708. This 
30 produces a summed brightness level for each pixel set associated with the 
bracketed images. 
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The process continues with step 710 which involves taking a count of the 
number of pixels exhibiting the same brightness level. It is believed that a better 
result is obtained if the summed brightness values are not averaged down to the 
5 original number of levels before histogram equalization. Therefore, the 
aforementioned brightness levels will correspond to the previously computed 
summed brightness values associated with the pixel sets. Fig. 8A presents a plot 
of a histogram for the summed brightness values associated with the images of 
Figs. 1 A through 1C. The histogram is constructed as follows. For each summed 
10 brightness level, the number of pixel sets having that summed brightness level 
are counted (as indicated above). In this case, each pixel set consists of three 
corresponding pixels from each of the three images of Figs. 1 A through 1C. In 
the histogram of Fig. 8A, these counts are plotted on the vertical axis against 
their associated summed brightness levels on the horizontal axis. 

15 

A cumulative distribution function is next computed using the summed 
brightness histogram (step 712). Referring to Fig. 8B, this cumulative distribution 
function can be visualized as follows. For each summed brightness level shown 
on the horizontal axis, the number of pixel sets at or below that level are plotted 

20 on the vertical axis. The cumulative counts are normalized in step 714 to make 
the maximum count correspond to the maximum summed brightness level. For 
example, in the case of the three images of Figs. 1A through 1C, the maximum 
count could be made equal to the maximum brightness level tripled. The 
normalized cumulative count value that corresponds to the summed brightness 

25 value associated with each pixel set is then identified in step 716. These 

identified cumulative count values are divided by the number of images in the set 
of bracketed images to produce a new pixel brightness value associated with 
each pixel set (step 718). In the example case, the identified cumulative counts 
would be divided by 3. 

30 

The cumulative distribution function is then used to generate the 
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aforementioned composite image. Specifically, as indicated in step 720, the 
composite image is generat d by using the new pixel brightness value associated 
with a pixel set as the pixel valu for a pixel in the composite image 
corresponding to the location of the pixels of the associated pixel set in the scene 
5 depicted by the images. 



4.0. Partial Equalization 

10 

A potentially advantageous variation on the full histogram equalization 
process described above is to only partially equalize the image. Partial histogram 
equalization is beneficial in that it can mitigate the effects of noise, as might be 
introduced by large areas of a single color in the images. The full histogram 
15 equalization techniques tend to stretch such a color to fill the whole gamut, and 
hence accentuate any noise. 

One way of accomplishing the aforementioned partial equalization is to 
blend the normalized cumulative distribution function with a straight line function. 

20 Referring to Figs. 9A through 9C, this blending operation is depicted graphically. 
Fig. 9A presents a graph of a cumulative distribution similar to that of Fig. 8B. A 
plot of a straight line function is shown in Fig. 9B. This function varies linearly 
from an origin to an end point at the maximum coordinates, which are the same 
for both axes. Fig. 9C depicts the result of blending the cumulative distribution 

25 with the straight line. 

While the foregoing blending operation does mitigate the effects of noise, 
it may not produce the optimum improvement in exposure and tone in the 
composite image. This situation can be avoided by weighting the blending 
30 operation so that the straight line function contributes less than 100 % to the 
combined function. Thus, the contribution of the straight line function to the 
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combined function could vary anywhere from 0% (i.e., no contribution) to 100 % 
(i.e., equal weighting). The choice of what particular contribution percentage 
should be used in the blending operation will depend on the images being 
processed, with the goal being to produce a composite image having the best 
5 possible improvement in exposure and tone uniformity in comparison to the 
bracketed images. For example, a scene depicted in the bracketed images that 
has large areas of a single color will typically require a greater contribution 
percentage from the straight line function in order to mitigate the resulting noise 
and produce the best possible improvement in the composite image. Whereas, 
10 images of a scene having fewer large single color areas will not require as much 
(if any) blending to produce the best possible improvement. 

In regard to the bracketed images provided in Figs. 1A through 1C, the 
results of the foregoing blending operation are shown in Figs. 10A and 10B. Fig. 
15 1 0A shows the composite image produced in accordance with the full histogram 
equalization approach, i.e., without any blending to mitigate noise. Fig. 10B 
shows the composite image produced using the partial equalization approach and 
a straight line function contribution percentage of 50 %. Notice the improvement 
in the appearance of the composite image in Fig. 10B. 

20 

5.0. Pixel Brightness Values. 

In the preceding description of the preferred embodiments of the present 
invention, the pixel brightness values were used as the basis for the processing 

25 of the bracketed images. However, there are more than one way to measure 
pixel brightness. If the images are black and white, then the standard pixel gray 
levels would preferably be used as the measure of pixel brightness. However, if 
color images are being processed, pixel brightness is measured differently. In 
the tested embodiments of the present invention, the red, green, and blue 

30 channels were handled as if they represented separate red, green and blue 
pixels and were all thrown into the same histogram. Thus, a single set of 
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correspondingly located pixels in the images contributed to the histogram counts 
three times, once for each color channel. This was done based on the 
assumption that the relationships between the response curves for each channel 
at different exposures are the same. 

5 

An alternate possibility would be to compensate each channel separately. 
The disadvantage of this approach, however, is that the hue in the image could 
be changed. Consider for example an image region that has a large variation in 
the red channel, and a small variation in the blue channel. Per-channel 
1 0 histogram equalization would stretch the blue channel more than the red one, and 
so the local hue would be changed. 

Another possibility would be to base the equalization process on the so- 
called perceived brightness. In a first version of this option, the standard Y- 

1 5 luminance channel used in video coding is used as the measure of pixel 
brightness. In another version, the new Y-channel values computed via the 
equalization process are used as a guide to modify the red, green, and blue 
channels. Specifically, the amount of compensation applied to the Y-channel 
associated with a given pixel (i.e., the ratio between the new value and the 

20 original value) is used to simultaneously correct the red, green, and blue 
channels of that pixel. 

6.0. Applications. 

25 Referring to the suitable computing environment depicted in Fig. 2, the 

preferred embodiments of the present invention could be implemented as part of 
a photographic manipulation software program. In addition, present invention 
could be embodied as special purpose software in newer generation digital 
cameras which include the capability to employ software programs. To this end, 

30 the camera could be configured so as to photograph a scene at different 

exposures in rapid succ ssion, and to then read out the different pixel values to 
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create the requir d bracketed images. 
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CLAIMS 

1 . A computer-implemented process for generating a composite image 
from a set of digital images of a static scene, which exhibits an improved 

5 uniformity in both exposure and tone in comparison to the set of images, said 
process comprising using a computer to perform the following steps: 

creating the set of digital images each of which is captured at a 
different exposure level; 

generating the composite image by setting each pixel in the 
10 composite image to a characteristic pixel value that influences the exposure and 
tone of the pixel based on the distribution of the characteristic pixel values of the 
pixels of each image in the set of images. 

2. The process of Claim 1 , wherein the characteristic pixel value is a 
15 pixel brightness level. 

3. The process of Claim 2, wherein the generating step comprises the 
steps of: 

generating a n-dimensional histogram from the brightness values 
20 associated with each pixel of each image in the set of images, wherein n is equal 
to the number of images in the set of images; 

fitting a curve to the histogram data; 
for each data point of the histogram, 

projecting the histogram data point onto a point of the curve 
25 which is nearest to the data point; 

identifying a new pixel brightness value associated with the 
point of the curve coinciding with the projected histogram data point, and 

associating the identified new pixel brightness value with the 
pixel corresponding to the projected data point; and 
30 generating the composite image by using the new pixel brightness 

value associated with each pixel of one of the images of the set of images as a 
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substitute for an original brightness valu of the pixel. 

4. The process of Claim 2, wherein the generating step comprises the 
steps of: 

5 computing a summed brightness level for each pixel set in the set of 

images by adding the brightness level exhibited by each pixel in a pixel set, 
wherein a pixel set comprises a pixel from each image of the image set which 
represents the same portion of the scene depicted by the set of images; 

dividing the summed brightness level computed for each pixel set by 

1 o the number of images in the set of images to produce an average brightness 

level; 

generating the composite image by using the average pixel 
brightness level associated with each pixel set as the brightness value for a pixel 
in the composite image corresponding to the location of the pixels of the pixel set 
15 in the scene depicted by the set of images. 

5. The process of Claim 2, wherein the generating step comprises the 
steps of: 

computing a summed brightness level for each pixel set in the set of 
20 images by adding the brightness level exhibited by each pixel in a pixel set, 
wherein a pixel set comprises a pixel from each image of the image set which 
represents the same portion of the scene depicted by the set of images; 

generating a cumulative distribution function based on a count of 
the number of pixel sets having the same summed brightness level; 

2 5 normalizing the cumulative distribution function so that a maximum 

cumulative count value thereof corresponds to a maximum possible summed 
brightness level; and 

generating the composite image by using the normalized cumulative 
distribution function to identify a cumulative count value that corresponds to the 
30 summed brightness level for each pixel set, dividing the cumulative count value of 
each pixel set by the number of images in the image set to produce a composite 
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pixel brightness level for that pixel set, and using the composite pixel brightness 
level associated with each pixel set as the brightness value for a pixel in the 
composite image corr sponding to the location of the pixels of the pixel set in the 
scene depicted by the set of images. 

5 

6. The process of Claim 2, wherein the generating step comprises the 
steps of: 

computing a summed brightness level for each pixel set in the set of 
images by adding the brightness level exhibited by each pixel in a pixel set, 
l o wherein a pixel set comprises a pixel from each image of the image set which 
represents the same portion of the scene depicted by the set of images; 

generating a cumulative distribution function based on a count of 
the number of pixel sets having the same summed brightness level; 

normalizing the cumulative distribution function so that a maximum 
15 cumulative count value thereof corresponds to a maximum possible summed 
brightness level; 

blending the normalized cumulative distribution function with a 
straight line function to produce a blended distribution function, said blended 
distribution function reducing the effects of noise on the distribution function 
20 caused by large areas of a single color in the scene depicted in the set of images; 

generating the composite image by using the blended distribution 
function to identify a cumulative count value that corresponds to the summed 
brightness level for each pixel set, dividing the cumulative count value of each 
pixel set by the number of images in the image set to produce a composite pixel 
25 brightness level for that pixel set, and using the composite pixel brightness level 
associated with each pixel set as the brightness value for a pixel in the composite 
image corresponding to the location of the pixels of the pixel set in the scene 
depicted by the set of images. 

30 7. The process of Claim 6, wherein the blending step comprises the 

step of assigning a weighting factor to the straight line function that determines a 
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degree to which the straight line function contributes to the blended distribution 
function. 



8. The process of Claim 7, wherein the weighting factor associated 

5 with the straight line function varies in a range which causes the contribution of 
the straight line function to the blended distribution function to vary anywhere 
from 0 percent to 100 percent. 

9. The process of Claim 2, wherein the set of images comprises black 
10 and white images and the pixel brightness level comprises a pixel gray level. 

1 0. The process of Claim 2, wherein the set of images comprises color 
images and the red, green and blue pixel channels of each pixel in the composite 
image are considered as separate pixels, and wherein the pixel brightness level 

1 5 corresponds to the color level of each of the pixels. 

1 1 . The process of Claim 2, wherein the set of images comprises color 
images and the pixel brightness level is modeled by the Y-luminance channel, 
and wherein the red, green and blue pixel levels of each pixel in the composite 

20 image are determined in accordance with the Y-luminance channel. 

12. A system for generating a composite image that exhibits an 
improved uniformity in both exposure and tone in comparison to a set of input 
images each of which exhibits a different exposure level, the system comprising: 

25 a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program 
modules of the computer program to, 

access said set of digital images; 
30 generate the composite image by setting each pixel in the 

composite image to a characteristic pixel value that influences the exposure and 
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tone of the pixel based on the distribution of the characteristic pixel values of the 
pixels of each image in the set of images. 

1 3. A computer-readable memory for causing a computing device to 
5 generate a composite image that exhibits an improved uniformity in both 

exposure and tone in comparison to a set of input images each of which exhibits 
a different exposure level, comprising: 

a computer-readable storage medium; and 
a computer program comprising program modules stored in the 
10 storage medium, wherein the storage medium is so configured by the computer 
program that it causes the computing device to, 

access said set of digital images; 
generate the composite image by setting each pixel in the 
composite image to a characteristic pixel value that influences the exposure and 
15 tone of the pixel based on the distribution of the characteristic pixel values of the 
pixels of each image in the set of images. 

1 4. A digital camera comprising: 

a computing device comprising a component for generating a 
20 composite image that exhibits an improved uniformity in both exposure and tone 
in comparison to a set of images captured by the digital camera; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program 
modules of the computer program to, 
2 5 cause the digital camera to create the set of digital images 

each of which is captured at a different exposure level; 

generate the composite image by setting each pixel in the 
composite image to a characteristic pixel value that influences the exposure and 
tone of the pixel based on the distribution of the characteristic pixel values of the 
30 pixels of each image in the set of images. 
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15. The digital cam ra of Claim 28, wherein the program module for 
causing the digital camera to create the set of digital images comprises a sub- 
module for causing the camera to take the images in rapid succession in 
response to a command by a user of the camera. 

5 

16. A computer-implemented image manipulation process for 
generating a composite image from a set of digital images of a static scene which 
were captured at different exposure levels, wherein the composite image exhibits 
an improved uniformity in both exposure and tone in comparison to the set of 

10 images, said process comprising using a computer to perform the following steps: 
accessing the set of digital images; 
generating the composite image by setting each pixel in the 
composite image to a characteristic pixel value that influences the exposure and 
tone of the pixel based on the distribution of the characteristic pixel values of the 
15 pixels of each image in the set of images. 
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